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Abstract 
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1  Introduction 


Consider  the  general  nonlinear  system 


F(x,y) 


/ 

V 


Mx  +  Ny  —  h 
XYe 


=  0,  (x,y)>0, 


(1.1) 


where  x,y,h,e  G  Rn,  M,N  6  R"x",  X  =  diag(x),  Y  =  diag(y )  and  e  has  all  components 
equal  to  one. 

We  call  the  following  set  the  feasibility  set  of  problem  (1.1): 


ft  =  {(z,y)  :  x,y  G  Rn,Mx  +  Ny  =  h,  (x,y)  >  0}. 

A  feasible  pair  (a:,  y)  G  ft  is  said  to  be  strictly  feasible  if  it  is  positive.  In  this  work  we  tacitly 
assume  that  the  relative  interior  of  fl  is  nonempty,  i.e.,  strictly  feasible  points  exist. 

Problem  (1.1)  is  sufficiently  general  to  include  linear  and  quadratic  programming  prob¬ 
lems,  and  linear  complementarity  problems.  Observe  that  \{  N  =  —I,  then  this  problem  is 
the  standard  linear  complementarity  problem  (LCP).  Moreover,  the  assumption  that  M  is 
positive  semi-definite  will  be  sufficient  to  guarantee  that  the  algorithms  under  investigation 
produce  well-defined  iterates  (Corollary  2.1). 

It  is  well-known  that  quadratic  programs  are  special  cases  of  linear  complementarity 
problems.  We  now  provide  a  somewhat  different  formulation  of  quadratic  programs  as  spe¬ 
cial  cases  of  Problem  (1.1)  instead  of  those  of  the  standard  LCP.  Consider  the  quadratic 
program  (QP) 

minimize  cTx  +  |  xTQx 

subject  to  Ax  =  b,  (1.2) 

x  >  0, 

where  c,  x  G  R",  b  G  Rm,  A  G  Rmxn(m  <  n)  and  has  full  row  rank,  and  Q  G  R"xn  is 
symmetric.  In  Corollary  2.1,  we  will  demonstrate  that  iterates  produced  by  the  algorithms 
under  investigation  are  well-defined  if  Q  is  positive  semi-definite  on  the  null  space  of  A.  In 
this  case,  it  is  well-known  that  the  problem  is  convex  and  the  first  order  conditions  are  both 
necessary  and  sufficient  for  optimality.  The  first-order  conditions  for  (1.2)  can  be  transformed 
into  the  form  of  (1.1).  To  see  this,  let  B  G  R(n-m)xn  be  any  matrix  such  that  the  columns  of 
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BT  form  a  basis  for  the  null  space  of  A.  The  first-order  conditions  for  the  quadratic  program 
(1.2)  are  (see  Dantzig  [1]) 


/ 


\ 


Ax  —  b 

AT\  —  Qx  +  y  —  c 
XYe 


0,  (x,y)  >  0, 


(1.3) 


where  A  and  y  are  the  dual  variables.  To  eliminate  the  dual  variables  A  from  the  above 
system,  we  pre-multiply  the  second  equation  by  the  nonsingular  matrix  [. AT  BT]T.  Noticing 
that  BAt  =  0,  we  obtain 


0  = 


A 

B 


(AT A  -  Qx  +  y  -  c) 


'  AAT A  -  A(Qx  - y  +  c )  N 
^  —BQx  -f  By  —  Be 


Since  AAt  is  nonsingular,  A  is  uniquely  determined  once  x  and  y  are  known.  Removing 
the  equation  for  A,  we  arrive  at  the  following  2n-dimensional  nonlinear  system  with  non¬ 
negativity  constraints  for  ( x,y ) 


Ax  —  b 

—BQx  +  By  —  Be 
K  XYe 


0,  (x,  y)  >  0. 


Clearly,  (1.4)  is  in  the  form  of  (1.1)  with 


A 

0 

b 

M  = 

-BQ 

,  N  = 

B 

and  h  = 

Be 

(1.4) 


(1.5) 


When  Q  =  0,  the  quadratic  program  (1.2)  reduces  to  a  standard-form  linear  program 
(LP) 


minimize  cTx 

subject  to  Ax  =  b ,  (1-6) 

x  >  0. 

Hence  (1.2)  also  includes  the  linear  program.  However,  because  of  the  importance  of  lin¬ 
ear  programming  in  optimization,  we  will  state  results  for  linear  programming  separately; 
fully  aware  that  they  are  special  cases  of  quadratic  programming.  We  have  shown  that  the 
framework  of  problem  (1.1)  is  quite  general. 
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The  objective  of  this  work  is  to  analyze  the  asymptotic  behavior  of  a  generic  interior- 
point  algorithm  for  solving  problem  (1.1).  More  specifically,  we  will  study  the  Q- convergence 
rate  of  this  general  algorithm.  The  issues  of  global  convergence  and  complexity  are  not  of 
concern  here. 

Recently,  Zhang,  Tapia  and  Dennis  [18,  see  Theorem  3.1]  established  a  Q-superlinear 
convergence  theory  for  a  class  of  primal-dual  interior-point  algorithm  for  linear  programming. 
In  this  paper,  we  extend  their  result  to  the  general  problem  (1.1)  and  therefore  extend  the 
result  to  quadratic  programming  and  linear  complementarity  problems.  In  spite  of  its  close 
connection  to  [18],  we  have  made  this  paper  self-contained. 

Given  u,  v  E  R"  and  tj  €  R,  We  will  use  the  notation: 

min(u)  =  min  [u];  and  min(u,  v,  rj)  =  min{min(u),  min(u),  rj}, 

l<t<n 

where  [it];  denotes  the  i-th  component  of  it. 

The  paper  is  organized  as  follows.  In  Section  2,  we  describe  a  general  interior-point 
algorithmic  framework  for  (1.1).  Then  in  Section  3,  we  present  our  superlinear  convergence 
rate  result.  Concluding  remarks  are  given  in  Section  4. 

2  Algorithm 

It  is  now  fairly  well  understood  how  a  class  of  interior-point  algorithms  can  be  viewed  as 
damped  Newton  methods  and  that  the  inclusion  of  the  logarithmic  barrier  term  (so-called 
centering)  can  be  viewed  as  perturbing  the  right-hand  side  of  the  Newton  system.  Indeed, 
Zhang,  Tapia  and  Dennis  [18]  focused  on  issues  concerning  how  fast  the  damped  Newton 
method  could  approach  the  Newton  method  (i.e.,  step-length  approaches  one),  and  how  fast 
the  perturbation  term  (barrier  parameter)  should  be  phased  out  so  that  the  fast  conver¬ 
gence  of  Newton’s  method  is  not  compromised.  Their  work  covered  linear  programming 
applications.  As  previously  mentioned,  the  objective  of  the  present  work  is  to  extend  a 
particular  nice  part  of  their  superlinear  convergence  theory  to  quadratic  programming  and 
linear  complementarity  problems.  Our  vehicle  for  accomplishing  this  objective  is  the  use  of 
the  general  problem  (1.1).  We  assume  that  the  reader  is  familiar  with  the  above  algorithmic 
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considerations  and  therefore  present  our  algorithmic  framework  with  no  further  motivation 
or  explanation. 

Recall  that  F(x,y )  is  given  by  (1.1). 

Algorithm  1  Given  a  pair  (x0,y0)  >  0.  For  k  =  0,1,2,...,  do 


(1)  Choose  a-, t  6  [0, 1)  and  rk  ( E  (0, 1).  Set  pk  =  o'kxfyk/n. 

(2)  Solve  the  following  system  for  (Axk,  Ayk): 


F'(xk,yk) 

(3)  Compute  the  step-length: 


^  Ax  \  /  0  > 

^  hke  j 


\  A»  / 


-F(xk,yk)  + 


Oik 


~Tk 


min(Afc  1  Aik,  Yk  1 A yk, -rk) ' 
(4)  Update:  xk+l  =  xk  +  a^Aa^  and  yfc+i  =  y*  +  ctkAyk. 


(2.1) 


(2.2) 


Notice  that  in  Algorithm  1,  we  do  not  require  that  the  starting  point  (xo,yo)  be  feasible. 
Also  notice  that  without  the  perturbation  term  pke  in  the  right-hand  side  of  (2.1),  the  search 
direction  (Axk,A yk)  is  the  Newton  step.  We  always  have  0  <  ak  <  1.  Moreover,  a*,  =  1  if 
and  only  if  min(Afc  1Axk:Yk  lA yk)  >  —rk.  We  should  expect  that  only  in  rare  cases  would 
the  full  Newton  step  lead  to  a  strictly  positive  iterate;  hence  we  should  expect  in  most  cases 
to  have  ak  <  1  where  ak  is  given  by  (2.2).  The  choice  rk  =  1  corresponds  to  allowing 
steps  to  the  boundary  of  the  positive  orthant  and  a  loss  of  strict  feasibility.  Therefore,  it  is 
natural  to  view  Algorithm  1  as  a  perturbed  and  damped  Newton’s  method.  We  see  that  if 
(x0,y0)  is  in  fl,  then  the  iteration  sequence  {(xk,  yk)}  will  be  strictly  feasible.  In  the  case 
of  linear  programming,  there  are  no  linear  equations  in  F(x,y)  that  involve  both  x  and  y. 
If  {xk,Vk)  e  0,  then  different  step-lengths  can  be  used  to  update  xk  and  yk  and  still  retain 
strictly  feasible  (xk+i,yk+i).  This  strategy  has  been  shown  to  be  more  efficient  in  practice 
(see  Lustig,  Marsten  and  Shanno  [9],  for  example).  However,  it  will  not  affect  our  results 
since  our  analysis  will  show  that  as  long  as  rk  — >  1  both  step-lengths  will  converge  to  one. 
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Algorithm  1  covers  or  is  closely  related  to  a  wide  range  of  existing  interior-point  algo¬ 
rithms  for  linear  programming,  quadratic  programming  and  linear  complementarity  prob¬ 
lems.  In  particular,  it  covers  most  of  the  existing  primal-dual  interior-point  algorithms 
for  linear  programming  as  well  as  quadratic  programming,  including  Kojima,  Mizuno  and 
Yoshise  [7],  Todd  and  Ye  [15],  Monteiro  and  Adler  [12,  13],  Lustig  [8],  Gonzaga  and  Todd  [2], 
Mizuno,  Todd  and  Ye  [11],  Algorithms  for  linear  complementarity  problems  that  are  cov¬ 
ered  by  or  closely  related  to  Algorithm  1  include  Kojima,  Mizuno  and  Yoshise  [5,  6],  Kojima, 
Megiddo  and  Noma  [3],  and  Kojima,  Mizuno  and  Noma  [4]. 

Although  these  algorithms  have  been  motivated  and  presented  in  various  ways  including 
path-following  (homotopy  or  continuation),  potential  reduction  or  affine  scaling  algorithms, 
most  of  them  fit  into  the  framework  of  the  perturbed  and  damped  Newton’s  method  applied 
to  the  general  problem  (1.1).  Due  to  the  extensive  activity  in  this  area,  our  list  of  references 
is  not  complete.  For  a  more  complete  list  of  references,  especially  in  the  cases  of  quadratic 
programming  and  linear  complementarity  problems,  we  refer  the  reader  to  two  recent  survey 
papers  by  Ye  [16,  17], 

The  following  proposition  gives  a  condition  which  guarantees  that  the  iterates  produced 
by  Algorithm  1  are  well-defined. 


Proposition  2.1  The  iterates  produced  by  Algorithm  1  are  well-defined  if  for  any  positive 
diagonal  matrix  D  e  Rnxn,  the  matrix  N  -  MD  is  nonsingular. 


Proof:  Since  (x0,y0)  >  0  and 


F'(x,y) 


M  N 
Y  X 

the  nonsingularity  of  F'(xq,  yo )  is  equivalent  to  that  of 


I  —MYf1 


F'(x0,y0)  = 


0  TV  —  MYfi~l  X0 
Yo  A'o 


(2.3) 


This  latter  matrix  is  nonsingular  if  and  only  if  N  —  MY0  1  Xq  is  nonsingular.  By  our  condition, 
(*i,yi)  is  well-defined.  An  induction  argument  completes  the  proof.  □ 

The  following  corollary  is  well-known  and  one  can  easily  verify  that  Proposition  2.1  is 
satisfied  in  the  three  cases  of  interest. 
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Corollary  2.1  The  iterates  produced  by  Algorithm  1  are  well-defined  for 

1.  the  linear  complementarity  problem  (N  =  —I)  with  M  positive  semi-definite , 

2.  the  quadratic  programming  problem  (1.2)  with  Q  positive  semi-definite  on  the  null  space 
of  A, 

3.  the  linear  programming  problem  (1.6). 

We  should  mention  that  we  have  stated  Algorithm  1  in  the  current  form  purely  for  the 
purposes  of  obtaining  a  unified  theory  and  notational  convenience.  By  directly  applying 
the  perturbed  and  damped  Newton  method  to  the  first  order  conditions  for  the  quadratic 
program  (1.2),  it  is  not  difficult  to  see  that  an  identical  iteration  sequence  {(xk,yk)}  will  be 
generated  without  eliminating  the  dual  variable  A  and  introducing  the  matrix  B. 

3  Superlinear  Convergence 

Ihe  literature  contains  numerous  studies  directed  at  investigating  the  convergence  properties 
of  interior-point  algorithms  covered  by  or  closely  related  to  Algorithm  1.  However,  most  of 
these  studies  were  concerned  only  with  the  issues  of  global  convergence  and  complexity.  The 
issue  of  convergence  rate,  which  is  certainly  important,  has  not  been  thoroughly  studied  for 
many  interior-point  algorithms.  One  of  the  few  papers  that  studied  asymptotic  behavior 
(local  convergence)  of  interior-point  algorithms  is  Kojima,  Megiddo  and  Noma  [3].  In  their 
paper,  Kojima,  Megiddo  and  Noma  proved  that  for  a  class  of  complementarity  problems, 
Q-linear,  in  addition  to  global  convergence,  superlinear  and  quadratic  local  convergence  can 
be  achieved  by  some  interior-point  algorithms  in  the  form  of  Algorithm  1.  However,  all  their 
convergence  rate  results  were  obtained  under  the  restriction  that  the  Jacobian  matrix  F'{x ,  y) 
was  nonsingular  at  the  solution.  In  this  section,  we  provide  a  set  of  sufficient  conditions 
for  superlinear  convergence  of  Algorithm  1  applied  to  the  general  problem  (1.1).  These 
conditions  do  not  require  the  nonsingularity  of  F’(x,y)  at  solutions.  How  to  apply  these 
conditions  to  construct  globally  and  superlinearly  convergent  algorithms  is  an  interesting 
topic  and  the  subject  of  further  research. 
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It  is  satisfying  that  it  is  possible  to  obtain  a  superlinear  convergence  rate  without  the 
assumption  of  nonsingularity  of  the  Jacobian  matrix  at  the  solution.  In  the  case  of  linear  pro¬ 
gramming,  this  allows  one  to  avoid  restrictive  nondegeneracy  assumptions.  The  motivation 
for  this  theory  came  from  numerical  experiments  that  demonstrated  superlinear  convergence 
even  for  highly  degenerate  linear  programs. 

At  the  fc-th  iteration  of  Algorithm  1,  let 

=  XkVk/n 
T)k  min^Tte)’ 

Since  xfyk/n  is  the  average  value  of  the  elements  of  XkYke,  it  is  clear  that  rjk  >  1. 


Theorem  3.1  Let  {( Xk,yk )}  be  generated  by  Algorithm  1  with  Tk  1  and  ak  —> ►  0,  and  let 
{xk,Vk)  — >  (z,,t/,).  Assume 

(i)  strict  complementarity, 

(ii)  the  sequence  {r]k}  is  bounded, 

(iii)  there  exists  p  €  [0, 1)  such  that  for  k  sufficiently  large 

AxjAyt  >  +  Ayl(XMYt-')Ayi,). 


Then  (a:*,?/*)  solves  problem  (1-1)  and  the  sequence  {F(xk,yk)}  component-wise  converges  to 
zero  Q-superlinearly.  Furthermore,  the  sequence  {F(xk,yk)}  is  Q-superlinearly  convergent, 
i.e.,  for  any  norm 


lim  sup 

A; — ►oo 


llF(a^+1,^+i)|| 

l|F(x*,Sfc)|| 


=  0. 


Before  we  prove  Theorem  3.1,  we  would  like  to  comment  on  the  assumptions  of  Theo¬ 
rem  3.1.  First,  Assumption  (iii)  is  not  particularly  restrictive  since  we  will  see  later  that 
in  the  context  of  linear  programming,  quadratic  programming  with  Q  positive  semi-definite 
on  the  null  space  of  A  and  linear  complementarity  problems  with  M  positive  semi-definite, 
we  have  the  stronger  result  that  AxfAyk  >  0  for  ( Xk,yk )  £  Ll.  We  used  the  more  general 
Assumption  (iii)  instead  of  AxfAyk  >  0  based  on  the  consideration  that  the  former  could 
be  useful  in  studying  situations  where  ( Xk,yk )  is  not  feasible.  We  stress  that  the  algorithm 
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designer  is  free  to  choose  ak  and  rk,  and  the  requirement  that  they  be  chosen  so  that  ak  — *  0 
and  Tfc  — >  1  is  not  particular  restrictive. 

On  the  other  hand,  the  compatibility  of  Assumptions  (ii)  with  the  choices  r*  — *•  1  and 
ak  — >  0  may  be  a  cause  for  concern.  It  seems  as  if  letting  t*  — >  1  and  <7*  — >  0  might  force 
rjk  — >  oo.  However,  our  numerical  experience  has  shown  this  not  to  be  the  case  for  linear 
programming.  In  our  numerical  studies  with  Netlib  problems  for  linear  programming,  we  let 
Tfc  — ►  1  and  ct k  — ►  0  and  always  observed  strict  complementarity  and  bounded  {rjk}-  While 
on  occasion  we  saw  some  rather  large  values  for  rjks,  they  eventually  leveled  off  or  actually 
started  to  decrease  as  the  iterates  approached  a  solution.  We  did  not  observe  continued 
growth  in  the  values  of  r/k  as  our  algorithm  converged.  Moreover,  the  observed  convergence 
was  clearly  Q-superlinear  and  cq t  — *■  1.  Of  course,  the  behavior  of  {rik}  varies  with  several 
factors  including  how  fast  {r^}  converges  to  one  and  {<7*,}  to  zero.  We  do  not  mean  to  imply 
that  unbounded  {i]k}  cannot  occur.  Instead,  we  feel  that  it  appears  to  be  more  the  exception 
than  the  rule  in  linear  programming.  It  still  remains  to  be  seen  whether  or  not  this  same 
phenomenon  exists  in  quadratic  programming  and  linear  complementarity  problems.  There 
is  no  doubt  that  this  topic  merits  further  study. 

To  prove  Theorem  3.1,  we  need  the  following  lemma. 

Lemma  3.1  Under  the  assumptions  of  Theorem  3.1, 

lim  ak  =  1.  (3-1) 

fc— >oo 

Proof:  Define  at  each  iteration 

pk  =  Xfl Axk,  and  qk  =  Yk~~lAyk.  (3.2) 

At  iteration  k,  from  (2.1)  and  (2.3)  we  have 

YkAxk  T  XkAyk  —  XkYke  T 

or  equivalently,  recalling  that  pk  —  crt-x-fyk/n  (see  Step  (1)  of  Algorithm  1) 

Pk  +  qk  =  -e  +  pk(XkYk)~1e  =  -e  +  crkTke,  (3.3) 

where  Tk  —  (xfyk/n)(XkYk)~1 .  Since  T]k  =  HTjteHoo,  Assumptions  (ii)  and  ak  — >  0  imply 

lim(p*  +  qk)  =  -e.  (3.4) 
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Multiply  both  sides  of  (3.3)  by  ( XkYk )  2  and  consider  the  square  of  the  t^-norm.  We 
have  the  following  equality 

\\{XkYk)hPk\\l  +  \\{XkYk)^qk\\l  +  2Axf Ayk  =  xTkyk  (l-2 <rk  +  af^k- —)  . 


n 


Note  that 


\\{XkYk)hpk\\l  =  Axl(X^Yk)Axk  and  \\{XkYk)h  qk\\\  =  &yTk{XkYkl)&yk. 


By  Assumption  (iii), 

(1  -  p)( \\(XkYt)ipk\\l  +  ||(X»n)*«t|II)  <  4»*  (l  -  art  + 


Dividing  the  above  inequality  by  xjyk/n ,  we  obtain 

(1  -  />)(lirr!«lll  +  II r;1®!!!)  <  »(i  -  2^  +  (3.5) 

_  1. 

Assumption  (ii)  implies  that  {||Tjfc||}  is  bounded  above  and  {||Tfc  2||}  is  bounded  away  from 
zero.  Therefore,  from  (3.5)  both  {pk}  and  {<?*:}  are  bounded.  It  now  follows  from  (2.2)  that 
{ak}  is  bounded  away  from  zero. 

Now  assume  [a:,],-  >  0.  Obviously, 


1  =  lim  =  lim  (1  +  ak\pk]i). 

k->  00  [Xfcji  k->  00 

This  implies  [pk]i  — ►  0,  because  {a*}  is  bounded  away  from  zero.  From  (3.4)  we  have 
[qk]i  — >  —1.  On  the  other  hand,  if  [x,],-  =  0,  then  [y„];  >  0  by  strict  complementarity. 
The  same  argument,  interchanging  the  roles  of  pk  and  qk ,  gives  [qk]i  — >  0  and  \pk\i  —■ y  — 1. 
Therefore,  the  components  of  pk  and  qk  converge  to  either  0  or  —1.  Consequently,  from  (3.2), 
(2.2)  and  rk  — >  1  it  follows  that  ak  — >  1.  This  completes  the  proof.  □ 

Now  we  are  ready  to  prove  Theorem  3.1. 

Proof  of  Theorem  3.1:  Let 


Fi(x,y)  =  Mx  +  Ny  —  h  and  F2(x,y)  —  XYe. 

We  will  prove  that  both  {Fi(xk,  yk)}  and  {F2(xk,  yk)}  component-wise  converge  to  zero 
Q-superlinearly.  This  will  imply  that  {F(xk,yk)}  component-wise  converges  to  zero  Q- 
superlinearly.  It  is  not  difficult  to  see  that  component-wise  Q-superlinear  convergence  of  a 
vector  sequence  implies  its  Q-superlinear  convergence. 
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First  we  show  that  {Fi(xk,yk)}  component-wise  converges  to  zero  Q-superlinearly.  If 
Fi(x0,  yo)  =  0  (i.e.,  (x0,  yo )  is  a  feasible  starting  point),  then  it  is  easy  to  see  that  Fi(xk,  Vk)  = 
0  for  all  k.  Therefore,  we  need  only  consider  the  case  where  Fi(x0,yo )  7^  0.  Note  that 
Newton’s  method  solves  linear  equations  in  one  step.  If  for  some  integer  p  >  0,  ap  =  1,  then 
we  have  Fi(xk,yk)  =  0  for  all  k  >  p.  Therefore,  we  need  only  consider  the  case  where  otk  <  1 
for  all  k.  It  is  easy  to  see  from  Steps  (2)  and  (4)  of  Algorithm  1  that 

Fi(xfe+i,i/fc+i)  =  ( Mxk  +  Nyk  -  h)  +  ak{MAxk  +  NAyk)  =  (1  -  ak)Fi(xk,yk). 

Since  ak  — ►  1,  {Fi(xk,yk)}  component-wise  converges  to  zero  Q-superlinearly. 

Next,  we  show  that  {^(rrfc,  yk )}  component-wise  converges  to  zero  Q-superlinearly.  From 
Step  (4)  of  Algorithm  1, 

Xk'xk+i  =  e  +  ctkPk  and  Yklyk+ 1  =  e  +  akqk- 

Adding  the  above  two  equations,  we  have 

Xk'xk+i  +  Yk~1yk+ 1  =  2e  +  ak(pk  +  qk)- 

It  follows  from  (3.4)  and  (**,—>  1  that 


lim  (Xk  1xk+i  +  Yk  1yk+i)  =  e.  (3.6) 

K— >00 

If  [x*],-  =  0,  then  by  strict  complementarity,  >  0  and  [yk+i]i/[yk\i  —• y  T  It  follows 
from  (3.6)  that  [xjk+1]t-/[x*],-  — ►  0.  Therefore,  [x*],-  — ►  0  Q-superlinearly.  By  the  symmetry  of 
the  relation  (3.6),  we  have  [yk\j  — ►  0  Q-superlinearly  if  [y*]j  =  0.  Thus,  all  variables  that 
converge  to  zero  do  so  Q-superlinearly.  That  is,  for  each  index  i  either 

lim  =  0  and  lim  =  1 

fc— 00  [xfcj,-  [yk\i 


or 


lim  =  1  and 

k~* 00  Ffcji 


lim 

k—*oo 


[yk+i]i 

[yk]i 


=  0. 


In  either  case,  for  every  index  i, 


,.  [^fc+iMyfc+i];  i-  [Xk+iYk+ie\i 
lim  — - — r- — - —  =  lim  — r  ^  'Vr~i —  =  6 
[xk\i[yk\i  [XkYke\i 


(3.7) 
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We  have  proved  that  {[Aftlte],}  converges  to  zero  Q-superlinearly  for  every  index  i.  As 
was  mentioned  above,  the  component-wise  Q-superlinear  convergence  of  {F(xk,yk)}  implies 
its  Q-superlinear  convergence.  This  completes  the  proof.  O 

A  key  idea  in  the  proof  of  Theorem  3.1  can  be  traced  back  to  a  work  by  Tapia  in  1980 
[14].  In  that  paper,  Tapia  pointed  out  [14,  Theorem  3]  that  an  algorithm  which  at  each 
iteration  satisfies  the  Taylor  linearization  of  the  complementarity  equation  has  the  property 
that  the  variables  that  converge  to  zero  do  so  Q-superlinearly.  This  result  assumed  strict 
complementarity  and  step-length  one.  Observe  that  (3.6)  is  equivalent  to 

NifYke.  T  Yk(x k-\-i  -Efc)  T  N.k(jjk- i-i  Vk)  *  0. 

We  see  that  the  Taylor  linearization  of  complementarity  is  satisfied  asymptotically  in  our 
situation. 

The  following  theorem  deals  with  the  Q-superlinear  convergence  of  Algorithm  1  applied 
to  linear  complementarity  problems,  quadratic  programming  and  linear  programming. 

Theorem  3.2  Let  {(xk,yk)}  be  generated  by  Algorithm  1  with  Tk  — *  1  and  a k  —*  0,  and  let 
{xk-,yk)  — +  (x*,y*).  Under  Assumptions  (i)-(ii)  of  Theorem  3.1,  if  (xp,yp)  £  Ll  for  some  p, 
then  (a :»,?/„)  solves  problem  (1.1)  and  the  sequence  {F(xk,yk)}  component-wise  converges  to 
zero  Q-superlinearly  for  the  following  three  cases: 

1.  the  linear  complementarity  problem  (N  =  —I)  with  M  positive  semi-definite, 

2.  the  quadratic  programming  problem  (1.2)  with  Q  positive  semi-definite  on  the  null  space 
of  A, 

3.  the  linear  programming  problem  (1.6). 

Proof:  We  need  to  prove  that  Assumption  (iii)  of  Theorem  3.1  is  satisfied  for  each  of  the 
above  three  cases.  Observe  that  for  all  k  >  p  we  have  ( Xk,yk )  €  0  and  MAxk  +  NAyk  =  0 
(see  (2.1)).  It  suffices  to  prove  that  uTv  >  0  for  all  u,v  6  R"  satisfying  Mu  +  Nv  —  0. 

In  the  first  case  ( N  =  —I),  Mu  +  Nv  =  0  is  equivalent  to  v  —  Mu.  Hence  uTv  — 
uT Mu  >  0  because  M  is  positive  semi-definite. 

In  the  second  case  (see  (1.5)),  Mu  +  Nv  —  0  is  equivalent  to  Au  =  0  and  BQu  —  Bv. 
Using  the  representations  u  =  BTU2  and  v  —  ATV\+BTV2,  where  iq  £  Rm  and  u2,v2  £  R"  m, 
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and  noticing  that  AT  _L  BT ,  we  have  uTv  =  u^BBTv 2.  Moreover,  BQu  =  Bv  is  equivalent 
to  BQBtu2  —  BBtv2.  Hence,  if  Q  is  positive  semi-definite  in  the  null  space  of  A,  then 

uTv  =  u\BBtv2  =  ul(BQBT)u2  >  0. 

The  third  case  follows  immediately  from  the  fact  that  Q  =  0  is  positive  semi-definite.  □ 

It  is  worth  noting  that  feasibility  is  assumed  in  Theorem  3.2  but  not  in  Theorem  3.1.  It  is 
not  clear  if  Assumption  (iii)  of  Theorem  3.1  may  be  satisfied  without  feasibility.  This  topic 
perhaps  deserves  more  study  because  infeasible  starting  points  are  used  in  most  practical 
implementations. 

4  Concluding  Remarks 

The  generality  of  problem  (1.1)  and  the  perturbed  and  damped  Newton’s  method  viewpoint 
have  enabled  us  to  analyze  the  local  convergence  behavior  of  a  class  of  interior-point  algo¬ 
rithms  for  linear  programming,  quadratic  programming  and  linear  complementarity  problems 
in  a  unified  approach. 

We  developed  a  Q-superlinear  convergence  theory  that  does  not  assume  any  information 
on  the  Jacobian  matrix  at  the  solution.  This  theory  was  used  to  establish  sufficient  conditions 
for  Q-superlinear  convergence  of  a  class  of  interior-point  algorithms  for  linear  programming, 
quadratic  programming  (with  Q  positive  semi-definite  on  the  null  space  of  A)  and  positive 
semi-definite  linear  complementarity  problems. 
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